Improving Discriminability Based Transfer by Modifying the Im Metric to Use Sigmoidal Activations Symposium: Artiicial Neural Networks and Adaptive Systems

We've previously described the Discriminability Based Transfer (DBT) algorithm, which improves back-propagation learning by using weights from networks that have been trained on related tasks. DBT evaluates the utility of source network hyperplanes on the new task using the Mutual Information (IM) metric. This paper extends DBT by relaxing its dependence on the assumption that hyperplanes with logistic (sig-moidal) activation functions tend to behave as thresholds. We show how the IM metric can be modiied to take sigmoidal activations into account. We present a new algorithm, called Sigmoidal Discriminability-Based Transfer (SDBT), and demonstrate its superiority to DBT.